Cache Efficient Bloom Filters for Shared Memory Machines

Bloom filters are a well known data-structure that supports approximate set membership queries that report no false negatives. Each element in the universe represented by the bloom filter is associated with k random bits in the structure. Traditional bloom filters, therefore, require k non-local memory operations to insert an element or perform a lookups. For very large bloom filters, these k lookups may require k disk seeks. Lookups can be expensive even for moderately sized filters which fit into main memory since k non-local memory accesses may result in L3, L2, and L1 cache misses. In this paper, we implement a cache-efficient blocked bloom filter that performs insertions and lookups while only accessing a small block of memory. We improve upon the implementation described by [4] by adapting dynamically to unbalanced assignment of elements to memory blocks. The end result is a bloom filter whose superior cache locality allows it to outperform a standard bloom filter on a shared memory machine even when it fits into main memory. This paper also surveys the design and analysis of three existing types of bloom filters: a standard bloom filter, a blocked bloom filter, and a scalable bloom filter. Ideas from these data structures will allow for the implementation of a cache efficient bloom filter which provides good memory locality. These data structures are used directly by our cache efficient bloom filter to obtain its properties.